ViT-SAPS: Detail-Aware Transformer for Mechanical Assembly Semantic Segmentation

Semantic segmentation of mechanical assembly images provides an effective way to monitor the process and improve product quality. Compared with other deep learning models, Transformer has advantages in modeling global context, it been widely applied various computer vision tasks including semantic segmentation. However, pays same granularity attention on all regions image, so some difficulty be images, which parts have large size differences information quantity distribution is uneven. This paper proposes a novel Transformer-based model called Vision Self-Adaptive Patch Size (ViT-SAPS). ViT-SAPS can perceive detail image finer-grained where locates, thus meeting requirements Specifically, self-adaptive patch splitting algorithm proposed split into patches sizes. The more region has, smaller into. Further, handle these unfixed-size patches, position encoding scheme non-uniform bilinear interpolation used after sequence decoding are proposed. Experimental results show that stronger ability than fixed size, achieves impressive locality-globality trade-off. study not only practical method for segmentation, but also much value application Transformers fields. code available at: https://github.com/QDLGARIM/ViT-SAPS.